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Summary  This  project  was  concerned  with  problems  in  operator  theory  and 
matrix  theory  that  underlie  the  maximum  entropy  principle  in  signal  proces¬ 
sing  and  systems  theory. 

♦  Two  papers  have  been  completed  that  generalize  the  maximum  entropy 
principle  to  a  wide  class  of  indefinite  hermitian  matrices. 

♦  New  work  on  orthogonal  polynomials  and  the  Gohberg-Semencul  formula 
for  the  inverse  of  a  Toeplitz  matrix  has  been  completed,  and  this  work  has 
stimulated  considerable  research  activity  by  other  authors. 

♦  Three  papers  have  been  completed  that  give  a  thorough  study  of  rank¬ 
preserving  extensions  of  band  matrices.  Factorization  theorems  are 
obtained  for  wide  classes  of  Toeplitz  matrices  and  Hankel  matrices,  and 
connections  are  given  to  the  Kalman  partial  realization  problem  in  signal 
processing.  Applications  are  are  also  given  to  signal  processing  when  the 
power  spectrum  consists  of  a  set  of  pure  line  spectra. 

♦  Three  papers  have  been  completed  that  generalize  for  rational  matrix 
functions  a  number  of  well-known  interpolation  problems  for  scalar  func¬ 
tions,  and  a  new  approach  to  tangential  interpolation  problems  is  presen¬ 
ted.  Applications  are  given  to  sensitivity  minimization  in  control  theory. 
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Applications  of  Operator  Theory 
to  Maximum  Entropy  Problems 

Robert  L.  Ellis,  Israel  Gohberg,  and  David  C.  Lay 

The  use  of  the  maximum  entropy  principle  (MEP)  or  method  (MEM)  is  well  known  in 
the  study  of  stationary  stochastic  processes.  Simply  stated,  the  MEP  is  used  to 
construct  a  covariance  function  of  a  process  in  a  situation  where  the  data  provides 
only  an  estimate  for  "part"  of  the  function.  The  technique  of  constructing  or 
"extending"  a  function  having  specified  properties  has  been  studied  in  a  variety  of 
cases  by  many  mathematicians,  beginning  with  Caratheodory  and  Toeplitz  in  the  early 
1900’s.  In  the  1970’s,  Burg  [4]  introduced  the  MEP  which  has  been  so  influential  in 
signal  processing.  We  have  analyzed  various  aspects  of  this  problem  in  5  publica¬ 
tions  (and  one  of  our  group,  I.  Gohberg,  has  coauthored  an  additional  19  papers  on 
the  subject  over  the  past  ten  years).  Until  recently,  research  has  focused  on 
either  (1)  functions  that  are  positive  and  matrices  (possibly  infinite)  that  are 
positive  definite,  or  (2)  functions  of  modulus  less  than  one  and  matrices  (or  opera¬ 
tors)  that  are  contractions.  Problems  in  these  two  areas  are  intimately  connected, 
and  solutions  in  one  area  often  may  be  translated  into  solutions  in  the  other  area. 

In  1986,  a  new  maximum  entropy  principle  for  contractions  was  published  in 
[5].  The  importance  of  this  principle  for  H°°-control  problems  was  discovered 
recently  by  the  research  engineers  Keith  Glover  and  John  Doyle,  who  described  in 
[13]  the  significance  of  maximum  entropy  in  the  problem  of  risk  sensitive  log 
control.  However,  they  also  showed  that  a  MEP  is  needed  for  extension  problems  more 
general  than  those  described  in  (1)  and  (2)  above. 

The  first  successful  attempt  to  broaden  the  MEP  in  a  finite-dimensional  setting 
to  indefinite  hermitian  matrices  was  done  by  our  research  group  [6].  The  work  con¬ 
tinued  in  n.3],  supported  by  the  AFOSR.  We  found  a  MEP  for  certain  broad  classes 
of  hermitian  band  matrices,  and  we  examined  some  cases  where  a  MEP  exists  only  as  a 
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constrained  maximum.  Our  work  explains  why  a  MEP  cannot  exist  for  all  hermitian 
matrices:  in  general  the  band  extension  of  a  hermitian  matrix  is  only  a  stationary 
point  for  the  determinant  function  and  can  yield  a  maximum,  a  minimum,  or  a  saddle 
point.  For  Toeplitz  matrices,  however,  the  general  selfadjoint  case  is  fairly  close 
to  the  positive  definite  case  that  arises  in  signal  processing  and  other  situations 
involving  stationary  stochastic  processes. 

When  the  power  spectrum  of  a  signal  consists  of  a  set  of  pure  line  spectra,  the 
autocorrelation  matrix  T(n)  is  singular  for  n  larger  than  some  N.  We  have  found 
that  the  higher  order  Fourier  coefficients  of  the  spectrum  are  completely  determined 
by  the  requirement  that  the  associated  extensions  of  T(N)  have  minimum  rank,  and  we 
have  an  explicit  formula  for  these  matrix  extensions.  Our  results  stem  from  a 
thorough  study  of  rank-preserving  extensions  of  band  matrices  [10,11].  Applications 
of  this  work  are  given  to  the  problem  of  extending  a  triangular  matrix  to  a  unitary 
matrix  or  a  contraction. 

A  factorization  formula  in  [10]  for  a  positive  semidefinite  Toeplitz  matrix  has 
been  generalized  in  [12]  to  a  wider  class  of  Toeplitz  matrices,  and  an  analogous 
formula  for  Hankel  matrices  has  been  discovered.  As  mentioned  in  [12],  the  Hankel 


case  is  connected  to  the  Kalman  partial  realization  problem  in  signal  processing. 

In  another  direction,  our  work  [9]  on  polynomials  orthogonal  on  the  unit  circle 
has  turned  out  to  be  of  wide  interest.  We  gave  a  new  proof  of  Krein’s  theorem  on 
the  distribution  of  the  zeros  of  orthogonal  polynomials,  a  result  that  generalizes 
the  standard  minimum  phase  theorem  for  error  prediction  filters.  We  showed  the  con¬ 
nection  between  Krein’s  theorem  and  the  Gohberg-Semencul  formula  for  the  inverse  of 


a  Toeplitz  matrix.  This  work  has  stimulated  at  least  four  other  papers  on  generali-  For _ 
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zations  and  related  topics  (see  [14]). 

As  we  pointed  out  in  uur  proposal,  some  of  Ihe  piobieius  v>e  are  studying  may  be  ',n _ 

approached  as  interpolation  problems.  In  the  past  two  years,  substantial  progress  _ 

has  been  made  in  this  area  by  Gohberg,  together  with  Ball  and  Rodman  [1,2,3].  They  ’n- - — 
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have  developed  a  new  approach  to  tangential  interpolation  problems,  including  the 
problem  of  reconstructing  a  rational  transfer  function  from  its  null  and  pole  data, 
and  the  problem  of  constructing  a  unitary  rational  function  when  only  a  part  of  the 
null  and  pole  data  are  given.  Problems  of  Nevanlina-Pick  type  are  also  considered. 
The  solutions  of  these  problems  are  given  in  an  explicit  form,  within  the  framework 
of  realization  theory  and  finite-dimensional  matrix  analysis.  Applications  are 
given  in  [3]  to  sensitivity  minimization  associated  with  stabilizing  compensators 
for  a  regular  plant,  an  important  area  of  interest  in  system  theory. 

Honors  and  Recognitions  In  August,  1988,  an  international  conference,  attended 
by  nearly  100  mathematicians  and  engineers,  was  held  at  the  University  of  Calgary  in 
honor  of  Israel  Gohberg’s  60th  birthday.  Ellis  and  Lay  were  among  the  27  invited 
speakers  at  the  week-long  conference.  All  three  investigators  have  been  invited 
speakers  each  year  at  national  or  international  conferences. 
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